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Abstract 

This study sought to determine whether GRE subscores (or GMAT) could predict 
graduation rates in related areas (math- oriented majors for GRE guantitative, etc.) in a 
sample of over 9,000 graduate students at a major public research university. Because few 
low guantitative scores were present in math- oriented majors, an attemp was made to 
determine whether thresholds occurred below which students would fail to graduate by 
grouping scorers (e.g. <400, 400-499, 500-599, 600+). Surprisingly, lower scoring groups 
(particularly <400) graduated at consistently and sometimes substantially higher rates 
than higher scoring groups. Possible reasons for this phenomenon are explored. 

Introduction and Purpose 

Because test makers generally recommend against using total scores in evaluation, 1 this 
study's primary purpose was to determine whether subtest scores on the GRE (guantitative 
and verbal) or total GMAT scores would prove better predictors of graduate student success 
than total GRE scores, and particularly so in fields emphasizing either guantitative methods 
(mathematics, physics, engineering, physical sciences, etc.) or language (English, 
communication, literature, history, etc.). Due to small samples for advanced graduate 
students, only beginning graduate students were included in this analysis. 

Background 

Recently, the status of graduate education has begun to receive national consideration. 
Lovitts & Nelson (2000) note: "..applications to graduate school in several disciplines have 
begun to decline— not only because well-paid high-tech employment is luring potential 
students away, but also because the news about the long-term collapse of the academic job 
market has finally penetrated the undergraduate culture." Perhaps as a result of these 
phenomena, greater attention is being paid to graduate retention today than ever before. Of 
course, one might assume that improved selection criteria would result in better graduate 
retention. One cheap (for the institution), readily available, and the fallaciously assumed- 
to-be-objective selection tool 2 is test scores. Therefore, considerable attention has been 
placed on these. However, findings indicate that they do not predict graduate success very 
well, if at all. 

Xiao (1998) in a 6-year study of 2,200 master's students at Northern Illinois University, 
found that incoming GRE had no effect except in engineering, where higher GRE scores 
associated with longer time to completion. 

Others reporting similar findings among graduate students include (Onasch, 1994; House 
and J ohnson, 1992 and Steinberg & Williams (1997). Onasch did find that higher 
undergraduate GPAs associated with less time to degree among graduate students. 
Arguably, one possible reason for the lack of relationship between test scores and success 
may be the restriction of range resulting from selection, however, statistics reported in the 



1 This was done in an earlier study showing no relationship between GRE or GMAT scores and graduate success 
(Micceri, 2000). 

2 For an extensive literature review on the lack of validity of tests, see: Micceri (2000). Why Do We Waste So Many 
Resources on Incredibly Costly and Practically Worthless Standardized Tests? - Available under Enrollment 
Management: Recruitment & Admissions: 4/2001 at http://usfweb.usf.edu/bpa/oir/studies.htm 
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Methods Section (Limitations) show this isn't really true for USF, and probably not any 
more true at the other Metropolitan Institutions where the studies above were conducted. 

Methods 

Because this study emphasizes analysis at the major level, and due to sample size 
limitations at that level, only beginning graduate students (seeking M.A. or M.S. degrees) 
were used in this study, and only those having an adequate time to earn a graduate degree: 
(1) for analysis at the college level, 1990 through 1996 cohorts were used, giving 2.67 years 
for the last cohort to earn a degree, and (2) for analysis at the major level, the 1990 through 
1997 cohorts were used, giving 1.67 years for the last cohort to earn a degree. 

For analyses at the college level, two random samples were drawn from the data to cross- 
validate results. 

In order to investigate the threshold issue (Does some point exist above which students 
must score to succeed in graduate school?), test scores were divided into four groups for 
GRE quantitative and verbal and GMAT. All students scoring below 400 were placed into 
one group; those between 400 and 499 in another, those between 500 and 599 in a third 
and finally, the highest scoring group included those scoring between 600 and 800. The 
lowest subscores were on the order of 200 . 

Limitations 

Although restriction of range may be a slight factor in reducing correlations between 
subtests and graduation, this did not appear much of a problem. Among the total 
populations upon which correlations were conducted, for the quantitative GRE subtest, 9% 
of all scores were below 400 and 20% were below 500, while 30% were 600 or above. For 
the verbal subtest, 12% were below 400 and 28% were below 500 while 16% were 600 or 
above. For both quantitative and verbal, minimum scores of 200 occurred more than once. 

The proportion of various cohorts lacking test data, largely because these data were never 
entered in the admissions files, ranged from 7% for the 1990 cohort to 24% for the 1997 
cohort. Although it is possible that the population lacking test scores may differ 
significantly from other students, and may therefore have somewhat altered relationships 
between tests and graduation, the fact that between 85% and 93% of most available cohorts 
had scores (only 76% in 1997), indicates that any such effects should be comparatively 
minimal. Also note that a substantial number of students who lack tests are foreign 
students, and that due to language biases inherent in these tests, scores for such students 
tend to be even less valid than do those for speakers of American Standard English. Thus, if 
their scores were present, we should most probably expect them to confound rather than 
clarify any relationships between test scores and graduation. 

Findings and Discussion 

Analyses were first conducted at the college level, then at the level of math and language- 
oriented majors. Figure 1 suggests that among math- oriented major fields, scores on the 
quantitative section of the GRE tend to be substantially greater than on the verbal section. 
However, there is, at best, miniscule evidence that higher scores on either the quantitative 
or verbal sections related to earning a degree. 3 In mathematics itself, interestingly enough, 
slightly higher scores on both verbal and math sections of the test associate with those 
earning degrees. However, these differences are neither significant nor important. 



3 Quan-Deg - represents the mean quantitative score for those earning degrees 
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particularly when one considers that the standard error of difference for these sub- tests is 
probably about 40 points (ETS, 2001). 

The bottom panel shows that students in the language- oriented majors are more likely to 
have balanced quantitative and verbal scores. Again, no differences occur between those 
earning and not earning degrees in any of the situations. In English itself, it appears that 
those scoring higher on both the quantitative and verbal sections are less likely to graduate. 
That this exactly counters the effects for mathematics suggests that any apparent 
differences are most likely random occurrences. 



The data investigated here do not suggest that GRE scores relate to success in graduate 
school for USF students admitted to specific programs, even in areas where one might 
expect them to show a specific relationship (e.g. mathematics or English). However, it 
should be noted that students in mathematics, both those who earned and who failed to 
earn a degree show somewhat higher mean quantitative scores than do students in the 
other sciences, and considerably higher quantitative scores than students in the language- 
oriented fields. A similar effect occurs for language- oriented fields, in which English majors 
show the highest verbal scores, while those in other language- oriented disciplines show 
somewhat lower scores. It could be that some threshold score exists below which students 
have difficulty. It is also possible that due to competition for spots, few, if any who score 
this low are ever admitted, thus, all who enter may be above such a threshold. 
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Figure 1 

Mean Quantitative and Verbal Comparisons for Math and Language- Oriented Majors - 

1990- 1996 Cohorts 



4 



Surprising Results from an Investigation of Threshold Possibilities 

In order to investigate this threshold issue, test scores were divided into four groups for 
GRE guantitative and verbal and GMAT. All students scoring below 400 were placed into 
one group; those between 400 and 499 in another, those between 500 and 599 in a third 
and finally, the highest scoring group included those scoring between 600 and 800. The 
lowest scores were on the order of 200 . 

T able 1 provides the total numbers of students with a given subtest (or GMAT) score, the 
percent in each of the four categories that graduated within a given college and the percent 
of all scores for that college that this group (e.g. between 400 and 499) represent. 

Interestingly, and surprisingly as well, the consistently higher graduation rates tend to 
associate with the lowest test score group ( < 40 0 ) rather than with the highest scoring 
group (600 to 800). This occurred in five of seven cases for the Quantitative GRE subtest 
(engineering lacked any cases below 400, so only seven cases occurred), six of eight times 
for verbal GRE and also for the GMAT. Thus, in 12 of 16 cases, the lowest scoring groups 
showed the greatest graduation rate. Rather than suggesting a threshold indicating success, 
these findings point in exactly the opposite direction: higher scores on tests past the point 
of 400 appear to reflect a threshold indicating a lower graduation probability. 

Again, interestingly and surprisingly, the lower scores (below 500), in most situations 
exhibited higher graduation rates than the higher scores (500 or above). It is not that any of 
these differences individually are important, but rather, the consistency of the effects that 
make this a meaningful finding. 



Table 1 

Graduation Rates Within Colleges by GRE Verbal or GMAT Group - 1 990 - 1 997 Cohorts 





N 


% Graduating 


% of All Taking Test 




All 


<400 


400-499 


500-599 


600 + 


<400 


400-499 


500-599 


600 + 


GRE - Quantitative 


Total 


7,514 


67% 


67% 


63% 


60% 


9% 


11% 


28% 


30% 


Architectur 


121 


0% 


51% 


38% 


37% 


2% 


15% 


33% 


34% 


Arts & Sci 


3,051 


61% 


62% 


62% 


57% 


10% 


14% 


30% 


24% 


Education 


1,900 


75% 


73% 


70% 


74% 


12% 


13% 


26% 


15% 


Ed Jnt Mjrs 


339 


48% 


52% 


52% 


53% 


9% 


15% 


38% 


22% 


Engineering 


969 




55% 


58% 


59% 


0% 


1% 


12% 


87% 


Fine Arts 


128 


61% 


80% 


65% 


55% 


18% 


8% 


27% 


17% 


Nursing 


406 


80% 


86% 


83% 


77% 


13% 


6% 


28% 


13% 


Pub Health 


600 


70% 


60% 


55% 


62% 


4% 


11% 


35% 


34% 


GRE - Verbal 


Total 


9,214 


73% 


64% 


59% 


61% 


10% 


15% 


31% 


16% 


Architectur 


121 


55% 


44% 


21% 


44% 


17% 


29% 


24% 


7% 


Arts & Sci 


3,052 


70% 


61% 


56% 


62% 


9% 


15% 


31% 


20% 


Education 


1,901 


81% 


73% 


69% 


67% 


15% 


14% 


23% 


11% 


Ed Jnt Mjrs 


339 


63% 


54% 


46% 


53% 


8% 


20% 


31% 


17% 


Engineering 


971 


71% 


57% 


59% 


50% 


17% 


19% 


24% 


15% 


Fine Arts 


128 


67% 


68% 


62% 


80% 


14% 


14% 


27% 


16% 


Nursing 


406 


82% 


84% 


83% 


81% 


10% 


8% 


28% 


12% 
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N 


% Graduating 


% of All Taking Test 




All 


<400 


400-499 


500-599 


600+ 


<400 


400-499 


500-599 


600+ 


Pub Health 


wm 


64% 


58% 


57% 


65% 


9% 


19% 


32% 


14% 


GMA1 


r 


Business 


1,696 


78% 


61% 


59% 


56% 


1% 


14% 


46% 


18% 



Thresholds at the Major Level for Quantitative-oriented and Language-oriented 
Majors 

Table 2 provides graduation rates and percentages of populations by major within college 
for guantitative- oriented majors on the guantitative GRE. Table 3 provides these data for 
verbal- oriented majors on the verbal subtest. In Table 2, in five of 13 situations the highest 
scoring group (600+) showed the highest graduation rates. On the other hand, in eight of 13 
situations, the lowest scoring group having any members showed the highest rate. Among 
the guantitative- oriented majors, mathematics had only guantitative scores of 500-599 or 
600+, and, those between 500-599 graduated at a greater rate than those at or above 600. 
In physics, there was a monotonic negative relationship between graduation rates and GRE 
guantitative scores, with the higher graduation rates occurring among lower scoring 
groups 4 . Students in mathematics education also showed a monotonic negative trend. Only 
in the college of engineering did any results equal expectations. In Computer Science, 
Electrical, Engineering Management, Industrial and Mechanical engineering, higher 
scoring groups graduated at a higher rate than lower scoring groups. However, for Chemical 
and Civil majors, the opposite held true. Thus, the results are clearly mixed for quantitative- 
oriented majors with eight having negative and seven positive trends associating with GRE 
subtest scores. 

Table 2 

Graduation Rates Within Departments by GRE Quantitative Group - 1990 - 1997 





Totals 


% Graduating 


% of All Students in Major 




All 


<400 


400-499 


500-599 


600+ 


<400 


400-499 


500-599 


600+ 


Arts & Sciences 


Chemistry 


20 


- 


1 00% 


50% 


53% 


0% 


< 1% 


10% 


85% 


Microbiology 


44 


1 00% 


50% 


50% 


61% 


2% 


7% 


32% 


52% 


Marine Science 


79 


- 


64% 


43% 


47% 


0% 


5% 


27% 


59% 


Mathematics 


36 


- 


- 


1 00% 


66% 


0% 


0% 


3% 


97% 


Physics 


46 


- 


1 00% 


75% 


54% 


0% 


< 1% 


9% 


89% 


Education Joint Majors 


Math Ed 46 - 


1 00% 


75% 


62% 0% 


< 1% 


35% 


63% 


Engineering 


Chmcl Eng 


48 


- 




67% 


50% 


0% 


0% 


13% 


88% 


Civil Eng 


226 


- 


1 00% 


62% 


61% 


0% 


< 1% 


21% 


78% 


Cmptr Eng 


40 


- 






45% 


0% 


0% 


0% 


1 00% 


Cmptr Sci 


151 


- 




44% 


47% 


0% 


0% 


6% 


94% 


Elec Eng 


233 


- 


50% 


58% 


62% 


0% 


< 1% 


8% 


91% 


Eng Mgt 


150 


- 


33% 


63% 


74% 


0% 


3% 


18% 


78% 


Indstr Eng 


40 


- 




0% 


59% 


0% 


0% 


3% 


98% 


Mech Eng 


74 


- 




33% 


56% 


0% 


0% 


8% 


92% 



4 Note - the 400-499 group had only one member, who graduated, thereby creating a 100% rate for that subgroup. 
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Table 3 shows that in only one of 13 possible situations (for English as a 2 nd language 
majors) among what might be termed verbal- oriented majors, did the highest scoring group 
on the GRE show the highest graduation rate. On the other hand, in two of nine situations 
the lowest scoring group (below 400) showed the highest graduation rates. Similar to 
mathematics, in 10 of 13 situations, students scoring between 500 and 599 graduated more 
freguently than did those with 600 or more. However, in only five of 13 cases did those 
scoring between 400 and 499 graduate at a greater rate than those scoring higher (either 
500-599 or 600 +). Further, in only three of nine cases did those scoring below 400 have 
the highest graduation rate. Specifically, English majors scoring between500-599 showed a 
higher graduation percentage than those at 600 or above. However, the 4% of English 
majors scoring between 400 and 499 on the verbal graduated less freguently. Among 
English as a second language majors, the two higher scoring groups (500-800) graduated at 
a greater percentage rate than did those who scored between 400 and 499; however, among 
Linguistic majors, those below 400 showed the highest graduation rates. As with the 
quantitative subscores, the results are mixed for verbal- oriented majors. 



Table 3 

Graduation Rates Within Departments by GRE Verbal Group - 1990 - 1997 







% Graduating 


% of All Students in Major 




N 


<400 


400-499 


500-599 


600+ 


<400 


400-499 


500-599 


600+ 


Arts & Sciences 


Communication 


39 




58% 


64% 


60% 


0% 


13% 


56% 


13% 


English 


104 




33% 


57% 


48% 


0% 


4% 


42% 


52% 


Engl 2nd Lng 


21 




64% 


1 00% 


1 00% 


0% 


19% 


19% 


29% 


French 


22 


20% 


63% 


50% 


33% 


23% 


14% 


27% 


14% 


History 


104 


0% 


46% 


30% 


59% 


1% 


14% 


41% 


31% 


Lib&lnfo Sc 


494 


82% 


80% 


79% 


76% 


8% 


7% 


28% 


31% 


Linguistics 


97 


60% 


41% 


56% 


46% 


10% 


21% 


26% 


29% 


Mass Comm 


147 


17% 


61% 


41% 


44% 


4% 


18% 


28% 


23% 


Philosophy 


35 


0% 


33% 


63% 


29% 


3% 


23% 


23% 


40% 


Education 


Reading Ed 35 67% 


50% 


88% 


0% 1 7% 


26% 


23% 


9% 


Education Joint Majors 


English Ed 


101 


60% 


59% 


42% 


43% 


5% 


16% 


36% 


21% 


Business - GMAT 


Economic 


24 


- 


43% 


67% 


25% 


0% 


17% 


38% 


33% 


Acctncy 


316 


0% 


71% 


70% 


63% 


< 1% 


9% 


48% 


19% 



Conclusion 

The findings of this study fail to suggest that GRE subscores relate to success in graduate 
school for USF students admitted to specific programs, even in areas where one might 
expect them to show a specific relationship (e.g. mathematics or English). However, it 
should be noted that few students were admitted to either mathematics or English degree 
programs without having at least a 500 score on the relevant GRE subtest. This effect 
stimulated an investigation of the possibility that threshold subscore values exist below 
which students in a given area of study could not perform without difficulty. Table 2 and 
Table 3, however, do not provide much support for the concept of a threshold, because. 
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where students were admitted at lower test levels, they tended to graduate at a higher rate 
than those with higher test scores. 

One admissions recommendation appears to result from this research, at least for those 
majors that are not either heavily language or math- oriented: 5 Completely reverse the 
criteria for admission to a graduate program. If you want higher graduation rates, then do 
not admit any students who score above 500 on a GRE subtest unless they are in a heavily 
language or math based subject, and in those subjects consider students scoring below 500 
unless your graduation rates appear different from overall trends (e.g. generally in the 
college of engineering, again, a lack of cases for reasonable comparison occurred) . In what 
might be termed "more balanced" fields, if admissions are limited to only students scoring 
below 400 on subtests, as opposed to those who score 500 or above, the gain in graduation 
rates ranges from a high of 19% on the GMAT, to a somewhat lesser 11% on the GRE 
guantitative, and finally, to a minimum of only about 4- 7% for the GRE verbal (Table 1 ) . 

One might ask: "How could this possibly occur if these tests have any validity?" Several 
possible explanation appear possible. 

First, all of the students included in this study had already completed a bachelor's degree 
prior to being admitted to graduate school. That a student would score below 400 on a GRE 
subtest suggests that they are poor test- takers. For the best and most intelligent students, 
poor test- taking ability can result from many factors, including, but not limited to a non- 
standard English background, severe test anxiety, specific learning disabilities, etc., etc. For 
such a student to accomplish an undergraduate degree, in which test- taking freguently is 
reguired for course success, we may probably assume that one of the following scenarios is 
true: 

• Such a student almost must exhibit a high degree of motivation, and almost surely 
persistence as well. Either of these two factors could contribute to higher rates of 
success than for those who have a comparatively "easy" time with tests (like me) . 

• Despite these poor subtest scores, all of these students were admitted to a graduate 
degree program, 6 which suggests that they received either strong recommendations, 
or possibly that departments apply more stringent acceptance criteria to such 
students, 7 either of which could relate to increased graduation rates. These results 
suggest that if criteria differ between high and low- scoring students, applying the 
same criteria to all students would tend to increase graduation rates. 

• It is conceivable that "better qualified" students are more likely to leave a master's 
program to take a job than those having what are generally considered to be 'lower 
qualifications." 

• If one assumes that standardized tests such as the GRE and GMAT have any 
validity 8 , then one might consider these data to indicate that many of USF's graduate 
programs better serve less academically qualified than more academically qualified 
students. 



5 A lack of cases from which to make judgements occurred for the heavily language or math-oriented majors. 

6 Admittedly, this may have occurred following evidence of their capabilities while being enrolled as 
unclassified/ non- degree- seeking students. 

7 Analyses failed to support the possibility that very high college GPA scores associated with such students. 

8 Almost all research in the vast testing literature indicates that such tests almost totally lack validity other 
than a low relationship with low-level undergraduate grades and a strong relationship with other such tests. 
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